1. Field of the Invention
The present invention relates to computer network analysis. More particularly, the present invention relates to a system and method for consolidating network streams for network analysis.
2. Background Information
In recent years, the popularity of intrusion detection systems (IDSs) has grown to the point where most Fortune 5000 corporate networks use some type of sophisticated IDS. An IDS operates using much of the same hardware as a packet capture device, as an IDS is, in fact, a type of packet capture device.
However, an IDS is generally not intended as a protocol analyzer, such as that typically used by network engineers to analyze network problems. Rather, an IDS locates known problems in the packet stream by applying, for example, sophisticated pattern matching algorithms. Based on the analysis, the IDS can generate appropriate signals or other alerts when some known condition exists. Such analysis is most often used to detect intrusion from the ingress traffic and/or breach of policy on ingress or egress traffic in a network (e.g., hackers).
As with many packet capture devices, an IDS may not function unless it has physical layer access to the network stream. In other words, the IDS can be physically inserted in the network stream, or, alternatively, a tap from the network stream can be presented to the IDS. FIG. 1 is a diagram illustrating a configuration of a network 100 in which individual IDSs 105 are connected to each of a plurality of network streams 111, 113, 117, 119, 121, 123, 127 and 129. Each of the plurality of network streams 111, 113, 117, 119, 121, 123, 127 and 129 comprises a communication or other suitable network link between network devices 142, 144, 146, 148, 152 and 154 for passing packet data or other appropriate information between such devices through the network. As illustrated in FIG. 1, each of the IDS 105 is in communication with a respective one of the plurality of network streams 111, 113, 117, 119, 121, 123, 127 and 129 using a corresponding tap 160.
Generally, it is appropriate for an IDS to reside as close as possible to the network firewall 170 and near the demarcation point to the outside (e.g., external network 103) of the given network 100. Indeed, it is useful if IDSs can reside on both sides (e.g., internal network 101 and external network 103) of the firewall 170. Many high-availability networks include an array constructed that will permit multiple paths for packets to transit the firewall 170 in both directions, dependent on such factors as, for example, network conditions and loading. Consequently, any packet can use any of the paths illustrated in FIG. 1 at any given time. Thus, time-synchronized IDSs would be needed on each link to be sure that all paths are covered. However, to perform time-sensitive coordinated pattern matching across eight or more links is a complex task.
For purposes of illustration, it can cost from $10,000 to $100,000 for an IDS system, and from $6,000 (for gigabit Ethernet) to $20,000 (for OC12) to $400,000 (for OC192) for each link covered. Assuming that the interfaces are gigabit Ethernet (e.g., 8*$6,000/interface=$48,000) and a “middle-of-the-road” IDS system (e.g., $50,000 for the eight IDSs 105 illustrated in FIG. 1) is used that is capable of handling the synchronization issues, an IDS solution for the network 100 illustrated in FIG. 1 would cost at least approximately $48,000+$50,000=$98,000. Because of the high costs involved, many companies might not elect to cover all of the accessible points in the network 100, especially in the external network 103.
Even if IDSs are installed as illustrated in FIG. 1 so that the user will know what is occurring outside the firewall 170 (e.g., in the external network 103), the IDS system can still suffer from a lack of synchronicity between packet capture devices. For example, if the time base used to timestamp the captured packets is not accurate, a skewed picture of the data will be received, which can result in a poor diagnosis of the network activity (e.g., missed detection of possible intrusions).